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1 Introduction 

In this paper we develop a part of the harmonic analysis associated with a convex cocompact 
subgroup r of a semisimple Lie group G of real rank one that could play the same role as the trace 
formula in the case of cocompact groups or groups of finite covolume. In these classical situations 
a smooth, compactly supported, and if -finite function / on G acts by right convolution Rr(f) 
on the Hilbert space L 2 (r\G). The trace formula is an expression of the trace of the restriction 
of this operator to the discrete subspace in terms of / and its Fourier transform /. The part 
involving / is called the geometric side and usually written as a sum of orbital integrals. The 
Fourier transform enters the trace formula in the case of non-cocompact subgroups, where it is 
combined with the scattering matrix. We will call this part the contribution of the scattering 
matrix. 

In the present paper we assume that T is a convex cocompact subgroup of G. Let X be 
the symmetric space of G and dX be its geodesic boundary. If T C G is a discrete torsion-free 
subgroup, then there is a T-equivariant decomposition dX = £1 U A, where A is the limit set of 
r. We call r convex-cocompact if T\X U is a compact manifold with boundary. 

Since the discrete spectrum of L 2 (r\G) is rather sparse we take the point of view that the 
the contribution of the scattering matrix is essentially (up to the contribution of the discrete 
spectrum) the Fourier transform of the geometric side of the trace formula. 

Thus our starting point is the geometric side. It is a distribution $ on G given as a sum 
of suitably normalized orbital integrals associated to the hyperbolic conjugacy classes of T (see 
Subsection |2.2| ). The objective of the trace formula in the case of convex cocompact T is an 
explicit expression for the Fourier transform of \F We are looking for a "measure" <3? on the 
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unitary dual G such that 

*(/)= IjMMdir) , (1) 



G 

where 9 n (f) := TV f(ir) is the character of tt. In the present paper we will formulate a precise 
Conjecture |5.2j about but we are not able to prove the formula (|l|) in the general case. 

The unitary dual G has a natural topology. Now observe that the intersection of the support 
of / and the support of the Plancherel measure of L 2 (r\G) is the spectrum of i?p(/). The Fourier 
transform of a compactly supported function / is never compactly supported. In order to do 
our computations we have to approximate Rr(f) by operators which have compact spectrum. 
The missing piece is some estimate which eventually allows for dropping the cut-off. Conjecture 



5.2 can easily be verified in the case of a negative critical exponent. 

In the present paper we will prove a formula which is similar to ([!]), but where \& has a 
different interpretation. Let R(f) denote the right-convolution operator on L 2 (G) induced by 
/. Then both, R(f) and Rr(f), have smooth integral kernels KR(f), K Rr rf\, and, by Lemma 



2.2, the value ^f(f) is nothing else than the integral 



*'(/) := / (K Rr{f) (g,g) - K R(f) (g, g))pL G (dg) . 
Jt\g 

We will show that can be applied to functions with compactly-supported Fourier transform, 



and our main Theorem 5^ is a formula 

*'(/) = / OMMd*) ■ (2) 



G 

together with an explicit expression for 

We will describe <3? in terms of multiplicities N-p(ir) and a density Lt(tt). If 7r is a representa- 
tion of the complementary series (a non-tempered unitary representation of G), then the integer 
iVr(7r) := $({-7r}) is the multiplicity of tt in L 2 (r\G). If tt is a discrete series representation, 



then by Proposition 6.14 iVr(7r) := <3?({7r}) is an integer. It is an interesting problem to study 
this number in detail. 

There are interesting operators with non-compact spectrum to which W can be applied. Let 
K C G be a maximal compact subgroup and O be the Casimir operator of G. We fix a -KT-type 
7 and consider (z — Q,)^ 1 on L 2 (r\G)(7) and L 2 (G)(7) if z is not in the spectrum. Let Kr(z) 
and K{z) denote the corresponding integral kernels. The difference K-p(z) — K(z) is smooth on 
the diagonal and goes into Vl/' if Re(z) <C 0. The consideration of these operators provides the 
link between the continuous part of $ and the Selberg zeta functions. In fact, using the analysis 
of the present paper we can show the meromorphic continuation of the logarithmic derivative 
and the functional equation (Theorem |6.12 ) of the Selberg zeta functions (By other methods we 



apriori know that the Selberg zeta functions are meromorphic, see below.). Our work extends 
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previous results || in the two-dimensional case and [11] in the spherical case of G = SO(l,n), 
n>2. 

Using symbolic dynamics of the geodesic flow and the thermodynamic formalism one can 



show that the Selberg zeta functions themselves are meromorphic functions of finite order [11]. 
This gives information about the growth of $ and on the counting function of resonances. It in 
particular shows that $ can be applied to Schwartz functions like the Fourier transform / of a 
i'T-finite smooth function / of compact support on G. 



2 The distribution * 

2.1 Invariant distributions 

Let G be a semisimple Lie group. We fix once and for all a Haar measure \xq on G. In this 
subsection we describe two sorts of conjugation invariant distributions on G, namely orbital 
integrals and characters of irreducible representations. 

Let 7 £ G be a semisimple element. The orbit 7 := {gjg" 1 1 g G G} of 7 under conjugation 
by G is a submanifold of G which can be identified with G~\G, where G 7 denotes the centralizer 
of 7. The inclusion z 7 : G^\G = 7 G is a proper map. Therefore the pull-back by z 7 is a 
continuous map 

i* : C C °°(G) -> C C °°(G 7 \G) . 

If we choose a Haar measure /ic 7 on G 7 , then we obtain an induced measure Hg^\g 011 G 7 \G 
such that 

/ f(g)VG{dg) = I / f(hg)fi Gl (dh)n Gl \G( d 9) ■ 

J G J G ry\Gr J G*y 

The orbital integral # 7 associated to 7 and the choice of the Haar measure \xq^ is, by definition, 
the composition of i* and the measure hg \g, i.e. 

fty(/) := PG 7 \G W) = / /(575 -1 )MG 7 \C?(^) ■ 

We now introduce the character associated to an irreducible admissible representation tt 
of G on a Hilbert space K / € C C °°(G), then 

:= / figMghaidg) 
Jg 

is a trace class operator on V n . The character 9 n is the distribution on G given by 



9M :=Trvr(/) 
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2.2 An invariant distribution asociated to T 

Let G be a semisimple linear connected Lie group of real rank one. We consider a torsion-free 
discrete convex-cocompact, non-cocompact subgroup T C G (see []|], Sec. 2). Let Or denote 
the disjoint union of manifolds G^\G, where 7 runs over a set CT of representatives of the set 
CT \ {[1]} of non-trivial conjugacy classes of T : 

Or = (J G 7 \G . 
7ecf 

The natural map ir ■ Or — > G is proper, and we obtain a continuous map 

i r : C C °°(G9 Cf (<5 r ) • 

For each 7 G CT we fix a Haar measure [xg 1 - Then we define a measure \xr on Or such that 
its restriction to G 7 \G is vol(r 7 \G 7 ) / u G ^\ vG . Note that this measure only depends on the Haar 
measure \xg and not on the choices of ■ 

Definition 2.1 The geometric side of the trace formula is the distribution ^ on G given by 

^ := fir ir ■ 
In terms of orbital integrals we can write 

*(/) = E voi(r 7 \G 7 )0 7 (/) . 

76CT 

Note that this distribution is in fact a measure, invariant under conjugation, and it only depends 
on T and the Haar measure fiQ. 

2.3 The Fourier inversion formula 

Let G denote the unitary dual of G. This is the set of equivalence classes of irreducible unitary 
representations of G equipped with a natural structure of a measurable space. For tt S G the 
operator vr(/) is, by definition, the value of the Fourier tranform of / at it, which we will also 
denote by f(ir). 

It is a consequence of the Plancherel theorem for G that there is a measure p on G such that 
for any / € C£°(G) and g G G we have 

f(g)= / Trvr^-VV)^) • 
Jg 

Note that p(dir) depends on the choice of the Haar measure \xq. Later in the present paper we 
will state a more precise version of the Plancherel theorem. 



2.4 The Fouriertransform of ^ 
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If h is a function on G such that h(ir) is a trace class operator on V n for almost all n (mod. 
p), then we form the function 

h(g) := / TrTr^)- 1 ^)^) 
JG 

if the integral exists for all g E G. 
2.4 The Fouriertransform of \1/ 

The contents of a trace formula for convex-cocompact groups T would be an expression of ^(f) 
in terms of the Fourier transform /. In other words, we are interested in the Fourier transform 
of the distribution ^. Since Vl/ is invariant this expression should only involve the characters 
9 n (f) = Tr/(7r). Thus there should exist a certain measure <1> on G such that the following 
equality holds true for all / £ C£°(G): 

*(/) = / OMMdn) . 
JG 

Note that there is a Paley- Wiener theorem for G which characterizes the range of the Fourier 
transform as a certain Paley- Wiener space. Apriori, $ is a functional on this Paley- Wiener space, 
and it would be a non-trivial statement that this functional is in fact induced by a measure on 
G. 



2.5 The distribution f as a regularized trace 

In the present paper we will not compute the Fourier transform $ of ^ in the sense of Subsection 



2.4 . Rather we will compute the candidate for $ using a different interpretation of ^. 

Let R denote the right-regular representation of G on L 2 (G). It extends to the convolution 
algebra by the formula 

R(f) = I f(g)R(g)» G (dg) . 



I G 

If / £ C£°(G), then R(f) is an integral operator with smooth integral kernel K R ^(g,h) = 
f{g^ 1 h). In a similar manner we have an unitary right-regular representation i?r of G on the 
Hilbert space L 2 (T\G) which can be extended to ^(G) using the formula 

Rrtf) = [ f{g)Rv{g)HG{dg) . 

JG 

If / £ C^°(G), then Rr(f) is an integral operator with smooth kernel 

K Rr{f) (g,h) = Y,f(9- 1 lh) . 

7£G 
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Indeed, for <j) S L 2 (r\G) we have 

Rtf)M = [ <i>(gh)f(h)» G (dh) 
Jg 

= I Hh)f(g- l h)^G(dh) 

Jg 



r \G 7er 



Lemma 2.2 For / G C C °°(G) we have 

*(/) = / [^ r(/ )(r 5 ,r<7) - K m (g,g)]t*G(dg) . 

Jr\G 

Proof. We compute 

*(/) = E voi(r 7 \G 7 )^ 7 (/) 



7GCT 

E vol(r 7 \G 7 ) f fig-^^aidg) 

E / fig^ig^Gidg) 

/ E E fig^h-^hg^aidg) 



E /to 1 ^g)VG(dg) 
[K Rr ( f )(g,g) - K R{f) (g,g)]fi G (dg) . 

r\G 

□ 

The expression of ^(/) in terms of the integral kernels of and Rr(f) can be used to define 
^ on other classes of functions or even on certain distributions. 

Using the Plancherel theorems for L 2 (G) and L 2 (T\G) the right-regular representations R 
and Rr can be extended. If / is infinite and / is smooth and has compact support, then we 
will see that g i-> [K Rr{f) (g, g) - K R{f) (g,g)} belongs to L 1 (T\G'), and thus 



*'(/) := / [K RAf) {g,g) - K R{f) (g , g)]» G (dg) 
Jr\G 



>t\g 

is well-defined. The main result of the present paper is an expression of ^'(/) in terms of / for 
those functions. 
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As mentioned in the introduction we are going to apply to the difference of distribution 
kernels of the resolvents (z — of the Casimir operator restricted to a -fT-type of L 2 (r\G) 
and L 2 (G), respectively. In this example the single kernels are not smooth, but their difference 
is so on the diagonal of T\G. Strictly speaking, the integral defining VI/' exists for Re(z) <C 0. 
For other values of z we introduce a truncated version ^>' R , R > 0, and we define the value of VP' 
as the constant term of the asymptotic expansion of ty' R as R — > oo. It seems to be an interesting 
problem to characterize the functions of f2 (restricted to a if -type) with the property that ^' R 
(applied to the corresponding distribution kernels) admits such an asymptotic expansion. 

Given a discrete series representation ir of G we can consider the corresponding isotypic 
components of L 2 (G) and L 2 (T\G). If we further consider a K-type of 7r, then the projections 
onto these components have smooth integral kernels. As a byproduct of the investigation of the 
resolvents we can show that can be applied to these integral kernels and that its values are 
integers. 

3 The Plancherel theorem and integral kernels 

3.1 The Plancherel theorems for L 2 (G) and L 2 (r\G). Support of Plancherel 
measures 

We start with describing the rough structure of the unitary dual G. First there is a countable 
family of square integrable unitary representations, the discrete series G^- The discretely de- 
composable subspace L 2 (G)d C L 2 {G) is composed out of these representations each occuring 
with infinite multiplicity. 

The orthogonal complement L 2 (G) ac of L 2 (G)d is given by a countable direct sum of direct 
integrals of unitary principal series representations. We are going to describe their parametriza- 
tion. Let G = KAN be an Iwasawa decomposition of G. The abelian group A is isomorphic to 
the multiplicative group R + . Let o and n denote the Lie algebras of A and N. Then dimjj(a) = 1 
and the roots of (o, n), fix an order on a. Let M = Zk{A) denote the centralizer of A in K. The 
unitary principal series representations 7r CT ' A of G are parametrized by the set (a, A) £ M x ia* . 
Let W denote the Weyl group Nk(A)/M , where Nk(A) denotes the normalizer of A in K. It 
is isomorphic to Z2, and we can choose a representative of the non-trivial element w £ Nk(A) 
such that w^ 1 = w. One knows that vr CT ' A is equivalent to ir' 7 ™' A , where a w denotes the Weyl 
conjugate representation of a given by a w (m) := a(m w ). For A / the representation 7r°" ,A 
is irreducible. If a is equivalent to a w , i.e. a is Weyl invariant, then it may happen that 
71"°"'° is reducible. In this case it decomposes into a sum -K a,+ © 7r cr ' _ of limits of discrete series 
representations. 
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The set of equivalence classes of unitary representations of G which we have listed above is 
the set of tempered representations. We refer to Sec. 8 of S for a discussion of the notion of 
temperedness for L 2 {G) and L 2 (T\G). 

The Plancherel theorem for L 2 (G) is, of course, explicitly known for a long time |7|]. The 
Plancherel measure p is supported on the set of tempered representations (compare |ffl]). In 
particular, it is absolutely continuous with respect to the Lebesgue measure on ia. Thus we 
can neglect the point A = 0. Then L 2 (G) ac decomposes as a direct integral of unitary principal 
series representations over M x ia* + with infinite multiplicity, and the Plancherel measure has 
full support. Note that the multiplicity space of the representation tt can be realized as V*. 

By G ac we denote the set of irreducible unitary principal series representations -ir a ' X , A ^ 0. 
The remaining unitary representations G c = G \ (Gd U G ac ) can be realized as subspaces of 
principal series representations 7r CT ' A with A G a+ U {0}. The case of limits of discrete series Gid 
(in this case A = 0) was mentioned above. The representations with parameter A > are not 
tempered and belong to the complementary series G cs . 

In U we studied the Plancherel theorem for L 2 (r\G). Let us recall its rough structure. The 
support of the corresponding Plancherel measure pr is the union of Gd, G ac , and a countable sub- 
set of G c . L 2 (T\G) decomposes into sum of subspaces L 2 (T\G) cusp , L 2 (r\G) ac , and L 2 (r\G) c . 
Here L 2 (T\G) cusp is discretely decomposable into representations of the discrete series, each 
occuring with infinite multiplicity, L 2 (r\G) c is discretely decomposable into representations be- 
longing to G c , each occuring with finite multiplicity, and L 2 (r\G) ac is a direct integral of unitary 
principal series representations with infinite multiplicity over the parameter set M x ia* + . On 
this set the Plancherel measure pr is absolutely continuous to the Lebesgue measure and has full 
support. The multiplicity space M v can be realized as a subspace of the T-invariant distribution 
vectors of V%, i.e., M w C r T4j-,-oo> where fr denotes the dual representation of tt. For tt G G ac we 



are going to describe M n explicitly in Subsection 3.3 



3.2 Extension of R and Rr 

The Plancherel theorem for G provides a G-equivariant unitary equivalence 

U:L 2 {G)^ f v:®V n p(dv) , (3) 
Jg 

where G acts on L 2 (G) by the right-regular representation R, and the action on the direct integral 
is given by 5 i — > {71" 1 — ^ idy* <8> tt(<?)}- We can identify V*®V n with the space of Hilbert-Schmidt 
operators on V n . For eft G C^°(G) we set 



U{<j)) := {tt - ^(vr)} , 



3.2 Extension of R and Rr 
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where <fi(g) '■= <f>(g 1 ). This fixes the normalization of the Plancherel measure p. 
The inverse transformation maps the family ir \— * h(ir) to the function 

U-\h){g) = [ Trir(g)h(n)p(chr) . 
Jg 

If / G L\G), than R{f) is given by g ^ {vr ^ idyj ® /(vr)}. 

The Plancherel theorem for T\G provides a G-equivariant unitary equivalence 

U r : L 2 (r\G) ^ / M V &V V pr(dvr) , (4) 

where G acts on L 2 (T\G) by the right-regular representation i?r, and the representation of G 
on the direct integral is given by g i— > {7r idju^ (8) 7r(fl)}- Again, if / € L 1 (G), than Rr(f) is 
given by 5 1— > {-7r 1— ► id^ <8> / 1 71 ")}- I n order to write down an explicit formula for Ur we first 
identify M* with and embed M n ®V n into Hom(M#, V n ). For G C£°(r\G) we define 

[/ r (0)(vr) := {M # 3 v » f faMg'^v u. G (dg) G V n } . 

Jt\g 

This fixes the normalization of pr ■ 

Let now h be a function on supp(p) such that /i(vr) is a bounded operator on V n . If h is 
essentially bounded (and measurable in the appropriate sense), then it acts on the direct integral 
([|) by ir 1— > {idy^* ® /i(vr)} and thus defines a bounded operator R(h) on L 2 (G) commuting with 
the left-regular action of G. 

In a similar manner, if h is a function on supp(pr) such that h{ir) is a bounded operator on 
V,,-, and h is essentially bounded, then it acts on the direct integral (|j) by ir {idjv^ <8> h(ir)} 
and thus defines a bounded operator Rr{h) on L 2 (r\G). 

Let us now assume that the h(ir) are of trace-class, and that J G \\h(n)\\i p(dTr) is finite, where 
||.||i denotes the trace norm ||^4||i = Tr \A\ for a trace class operator AonV n . Then R(h) is an 



integral operator with integral kernel K^r h \(g,k) = h(g k). Indeed, for <fi G G£°(G) we have 



(R(h)<P)(g) = f Tr7r(#)%)0(7rM^) 
Jg 



IG 

Tr f ■n{g)h{^{k- 1 )^(k)^ G (dk)p{dTr) 
Jg 

Tr Tr(k- 1 g)h{-K)p(dT:)(t)(k)n G (dk) 
G JG 

h(g- 1 k)0(k)u. G (dk) . 
G 



We are looking for a similar formula for the integral kernel of Rr(h) in Subsection 3.3. 
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3.3 The absolute continuous part of L 2 (T\G). Integral kernels for R r (h). 

In this subsection we describe in detail the Plancherel decomposition of L 2 (r\G) ac - The goal is 
to exhibit a class of functions ir h(ir) with the property that Rr(h) is an integral operator. 

Let P = MAN be a fixed parabolic subgroup. If (a, A) G M x o^, then we define the 
representation a\ of P by a\(man) := a(m)a p ~ x , where p G a* is given by 2p(H) = tr ad(H)\ n , 
H G o, and for A G and a = exp(iT) G A we put a A = e x ( H \ We realize the principal 
series representation H a,x := V^ a ,\ as the subspace of C~°°(G xp V ax ) of functions such that 
f\K ^ L 2 (K x jy V^r). Then ff££, = C ±0 ° (Gxp V ax ) are the spaces of smooth (resp. distribution) 
vectors of ir cr,x . By restriction to K we obtain canonical isomorphisms H a ' X = l?(K Xm^). It 
therefore makes sense to speak of smooth functions / on ia* such that /(A) G H a,x . 

Note that cWT can be identified with G/P. Let G/P = U A be the T-equivariant decom- 
position of the space G/P into the (open) domain of discontinuity f2 and the (closed) limit set 
A. As a convex-cocompct subgroup T acts freely and cocompactly on f2. We put B := T\Q. 
Furthermore, we define the bundle Vb(&\) —* B by Vb(<J\) ■= F\(G x p V ax )\Q. If A G ia*, then 
we have a natural Hilbert space L 2 (B, Vb(ct\)). Again, fixing a volume form on B we obtain 
identifications of the spaces L 2 (B,V B (a\)) with the fixed space L 2 {B ,Vb{(Tq)) so that it makes 
sense to speak of smooth functions / on ia* such that /(A) G L 2 (B, Vb((T\)). We refer to |J, 
Sec. 3, for more details. 

In U we defined a family of extension maps ext : L 2 (B,Vb((T\)) — > r lT^ ) . For A G ia*, 
A ^ 0, the extension map provides an explicit identification of the space of multiplicities M^ a ,\ C 
r #!- A with L 2 (B,V B (t-x)). 

The Plancherel measures p and pr on {a} x ia*^ are given by dl ^^ - p a (X)dX, where p a 
is a smooth symmetric function on ia* of polynomial growth (see ||, Lemma 5.5. (3)), and 
u>x '■= hma^oo a~ 2p vo\Q/K{KaK) (see ||, Sec. 11). Note that dX is the real Lebesgue measure 
on ia. 

We now describe the embedding 

! . dmu%) /• L2( Bi vb(or_ A ))®H ff .V(A)dA-.L 2 (r\G)«c • 

27TLUX J{cr}xia* + 

If v (8) w G L 2 (B, Vb(ct-a)) <8> -ffS) A , then we define < v (g> u> >:= (ext(v),w). Let be a 
smooth function of compact support on ia+ U {0} such that <p(X) G L 2 (B, Vb(<t_a)) <8> then 
we have 

^fVXs) = < (1 ® ^ x (g))<p(X) > Pa (X)dX . 

Note that ext may be singular at A = 0. In this case it has a first-order pole and p a (Q) = (see 
||, Prop. 7.4) such that the integral is still well-defined. 



3.3 The absolute continuous part of L 2 (T\G). Integral kernels for Rr(h). 
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We now fix a i^-type 7 G K. Let H a ' X {^f) denote the 7-isotypic component of H a,x . By 
Frobenius reciprocity we have a canonical identification 

H a ' X {j) = F 7 ® Hom^(V 7 , £T' A ) ^ y 7 ® Hom Af (y 7 , T/ CT ) . (5) 

Here a~ 1 (v0U) := U(v) and 0(U)(v) := U(v)(l). Any operator A G End(V 7 (8> Hom M (l / 7 , V^)) 
gives rise to a finite-dimensional operator F(A) G End(-£f°"' A ) which is trivial on the orthogonal 
complement of H a,x {^). 

Let g be a smooth function of compact support on M x za* such that q(o~, A) G End(V 7 ® 
Homjvf(^7, Kr))- We call g symmetric if it is compatible with the equivalences J"\ : H a ' X — > 
H aW ~ x , i.e. if F(q(a w , —A)) = J^" A o F(q(a, A)) o (J™ A ) -1 . If q is symmetric, then we can define 
the function /i 9 on G such that h q (7r) G End(V r 7r ) by h q (ir a ' X ) := F(q(a, A)) for (a, A) G M x ia* , 
A / 0, and by h q (ir) = for all other representations. 

Let 7r* : -?/Sd A - ► L 2 (B, Vb(°a)) denote the push-down map which can be considered here as 
the adjoint of the extension ext : L 2 (B,Vb{<7-\)) — > H°^^ . The composition 

7r <T ' A ( 5 )/ig(7r ,7 ' A )7r ,T ' A (fe- 1 )ext vr* 

is a finite-dimensional map from -ffS, A to H^, X . It is therefore nuclear and has a well-defined 
trace. 

Lemma 3.1 The operator Rr(h q ) has a smooth integral kernel given by 

K Rr(h q )(9,k) = £ ^^£ T r^ A ( 5 )/ i(/ (^ A )^ A (fc- 1 )e a ;tovr^ CT (A)dA . 

Proof. First of all note that the integral is well defined at A = 0. If ext o 7r* is singular at this 
point, then it has a pole of at most second order. But then the Plancherel density vanishes at 
least of second order, too. 

Let (f> G C£°(r\G). In the Plancherel decomposition it is represented by the function 
7r 1 — ^ C/r(0)(vr) G M n ®V n . We fix A for a moment and choose orthonormal bases {vi} of 
L 2 (B, Vb(ct-a)) and {w^} of i? CT ' A consisting of smooth sections. Furthermore, let {v 1 } and 
{w 3 } be dual bases of L 2 (B, Vb{<t\)) and H a '~ x , respectively. Then we have 

U r ((p)(TT a ' x ) = V f <v*® n a ^ x (k)w j > (p{k)iL G {dk) Vi ® . 

We have 

Ar(h q )(m9) = E / < ( X ® **'^)M* ff,A )) W)(tt*' a ) > p CT (A)dA . (6) 

o-eM 



12 



4 POISSON TRANSFORMS AND ASYMPTOTIC COMPUTATIONS 



We now compute 



< (l0^ A ( 5 )/ i ,(7r CT ' A ))[/r(0)(7r CT ' A ) > 

= [ <vi ® K a '- X (k)w j > <f>(k)n G (dk) < (1 ® TT^ X (g)h q (TT a ' X ))Vi ® Wj > 



I -J 



G 



G 



^2 < v { ® w j > 0(fc) < (1 ® ^ <T ' A ( 5 )/i g K' A )^ fT ' A (A;- 1 ))^ ® ^ > na{dk) 
G i,j 

^«/»(fc)(t;S^^)( e xt(t> i ),7r CT ' A ( 9 )/ l9 K' A )7r CT ' A (fc- 1 )u; j ) MG (^) 
i,3 

<t>(k){ext o vr,(u; J '), vr' J ' A ((7)/i (? (7r CT ' A )^ A (A:- 1 )u; j )/i G (dfc) 

j 

0(fe)<u^, vr <T ' A ( 5 )/i (? (^ <T ' A )^ <T ' A (A:- 1 )ext o n*Wi)fi G (dk) 

i 

(j)(k)Tv ^ x {g)h q {^ x )^ x {k~ l )ext o ^ HG(dk) . 



G i,3 



G 3 



G 3,1 



G I 



Inserting this computation into (||), the we obtain the desired formula for the integral kernel of 
R T {h q ). □ 



4 Poisson transforms and asymptotic computations 
4.1 Motivation 



Let q be symmetric and define and h q as in subsection |3,3| . We want to show that the function 
9 ^ i K R r (h q )(9,9) ~ K R(h q )(9,9)] belongs to L^I^G). It follows that 

= f T J K Rr(h q ){Wg)-K k{hq) {g,g)^ G {dg) 

is well-defined, and we are asking for an expression of fy'(h q ) in terms of q, respectively h q . 
In the present section we show related results using the language of Poisson transformations. 



In Subsection |5.l| we will provide the translation of these results and solve the problems above. 



4.2 Poisson transformation, c-functions, and asymptotics 

We fix a if-type 7 and a M-type a. Let T G Homj^(T4-, Vy) and A G a^. If w € Vy, then 
by Frobenius reciprocity we consider w (8) T* as an element of H u ~ x {^) which is given by the 
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function k i— ► T*('j(k^ 1 )w) under the canonical identification <p_\ : ^ C°°(K Xm Vg). 

We further put <I>a,^ := fix 1 ° <$>n- We will also use the notation $o,A for <f)\. 
The Poisson transformation 

:H^^C°°(Gx K V y ) 
is a G-equivariant map which is defined by the relation 

(w,P^m 9 )) ={w®T*^\g- 1 )^) , 

for all if) G H-oo, 

For the definition of the function c CT we refer to ||, Sec. 5. We have the relation 

c a (\)ca(-\)= Pa (\)- 1 . 

It turns out to be useful to introduce the normalized Poisson transformation °P^ := c CT (— \)~ l P^. 
We introduce the family of operators 

by 

rUm) :=°Pl(f)(ka) . 

In order to discuss the asymptotic behaviour of V\ a as a — > oo we need the normalized 
Knapp-Stein intertwining operators 

Note that J^ A o J™, _ A = id. We again refer to ||, Sec. 5, for more details. The following is a 
consequence of ||, Lemma 6.2. Let a G o* denote the short root of (a, n). For A G ia* we have 

Via = a x ~P^-T o <D 0jA + a~ x -^(w)T o <K 0j _ A o J^ A + a-^tt-oc (A, a) , (7) 

where IZ-oo (A, a) o $ A q remains bounded in C°°(ia*, Rom(H^, C~°°(K x M V 7 ))) as a — > oo. 
Multiplication by T (resp. j(w)T) is here considered as a map from .fff^ (resp. H^^) to 
C~°°(K Xm V-y) in the natural way. If x, x are smooth functions on K/M with disjoint support, 
then x^— oo(A, o)x ^a.o remains bounded in C°°(ia*,Hom(iI^, C°°(K Xm Vj))) as a — > oo. 
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4.3 An estimate 

In order to formulate the result appropriately we introduce the following space Cr{G) of functions 
on G. For each compact V C ft and integer ./V we consider the seminorm 

\<P\ v ,n := sup (1 + | log(a)\) N a 2 P\<j>(kah)\ , </> G C(G) . 

kahdVA+M 

Here we consider V as a subset of K using the identification G/P = K/M. We define the 
Frechet space Cr{G) as the space of all continuous functions <j) on G such that |0|y,Ar < oo 
for all compact V C ft and N G N. If is T-invariant and belongs to Cr(G), then clearly 
4>eL\F\G). 

Now let T G Hom M (V a , V y ), R G Hom M (V^, V^), and g G C%°(ia*). Then we can define the 
operator 

A q = A q (T, R) := / °Pfo(ext o vr* - l)o(°P« A )*g(A)dA G Hom(C-°°(Gx^ 7 ), C°°(Gx^ 7 )) . 

This operator has a smooth integral kernel (g, h) *— > A q (g, h) G End(V^,). The main result of the 
present subsection is the following estimate. 

Proposition 4.1 

LV^I GCr(G) . 

Proof. Note, that we only have to show finiteness of the norms | -| v,jv, where V C £1 is compact 
and has the additional property that V is contained in the interior of a compact subset V\ C £1 
satisfying jVi PI V\ = for all 1 ^ 7 G T. Indeed, any seminorm of Cr(G) can be majorized by 
the maximum of finite number of these special ones. 

We choose a smooth cut-off function xonfi such that supp(x) C V\ and supp(l — %) n V = 0. 
We further choose a compact V2 containing V in its interior and being contained in the interior 
of Vi, and a cut-off function \ on ^ suc h that supp(x) C V2 and supp(l — %) n V = 0. 

Then we can write for k G V 

°Pj o (ert o 7T* - 1) o (°P R x )*(kah, kah) 

= m-\(k)o[Vl a o(exto^-l)o(V R X:a r](k,k)o X (k) 7 (h) . (8) 

In order to employ the off-diagonal localization of the Poisson transformation we write 

X o Via o (ext o tt* - 1) o {V\ a Y o x 

= X°Vl a o(l-x)o(exto^-l)o X o(V R XA y o X (9) 
+X o Via o (ext o 7T* - 1) o (1 - x) o (V\ a y o X ■ 



4.3 An estimate 



15 



In @ we could insert the factor (1 — x) since % ° ( ex ^ ° 7T * ~ 1) ° X = 0- Using that supp(x) n 
supp(l — x) = we have 



X o PL o (1 - x) = a- x -P X o l{w)T o <D 0i _ A o J™ A o (1 - x) + a^Tl^X, a) , 



where TZ^X, a) o remains bounded in C°°(ia*, Hom^, C°°(K X M Vy))) as a — > oo. 
We obtain 

X o Vl >a o (ext o tt* - 1) o (T^J* o x = 

a" 2 ^ 2A X o j(w)T o d> ,- A o J™ A o (1 - x) ° (ear* o tt* - 1) o X ° $A,o ° R* ^7^ o X (10) 
+a"^x o 7(to)T o $ 0j _ A o J- A o (1 - x) o (ext o tt* - 1) o x o ( J^_ A )* o $_ A , ° R*j(w)*(m 
+a" 2 V A x o ^t^tT o $ (ext o tt* - 1) o (1 - x) o (Jg_ A )* o d>_ Ai0 o iT 7 H* o x (12) 
+a- 2 ^x ° 7(^)r o $ 0j _ A o J- A o (ext o tt* - 1) o (1 - x) o ( Jg_ A )* o $_ A>0 o iT 7 M* o x(13) 
+Q(A,o) , 

where a 2p+a Q(X, a) remains bounded in C°°(ia*, Rom(C-°°(K x M V y ), C°°(K x M F 7 ))) as a -> 
oo. 

We further compute using that the intertwining operators commute with ext o 7T* (compare 
the proof of Lemma [4.2| for a similar argument), the functional equation of the intertwining 
operators, and x ° (ext o 7T* — 1) o x = 

= a- 2 P X o 7H^ ° *o,-A <A (ear* o tt* - 1) o x o ( J^_ x )* o *_ A)0 iT 7 H* o X 

+a- 2 ^x o j(w)T o <D 0i _ A o J- A o (ext o tt* - 1) o (1 - x) o ( J^_ x )* o c5_ Ai0 o iT 7 H* o x 
= a- 2 P X o 7H^ ° *o,-A ° (ext o tt* - 1) o J™ A o ( JF_ A )* o $_ Aj0 o R*j(w)* o X 
= aT 2p X ° 7('w)7 1 ° ^0,-A ° (ext o 7r* — 1) o <I>_ A) o o i?*7(u>)* o x 
= a~ 2p j(w)T o $o,-a ° X (ext o 7r* — 1) o x o <5_ Aj o ° R* ; y(w)* 
= . 

Note that in (0) and © one of the intertwining operators is localized off-diagonally. We 
conclude that the following families of operators 

X o 7 (w)T o $o — a ° J'aX ° (1 - X) (ext o vr* - 1) o x o $ A0 o fi* ^L_^_ x 
X o ^§y T *0,A o (ext o tt* - 1) o (1 - x) o (J^_ A )* o $_ A Q o i?* 7 H* o X 
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belong to C°° (ia* ,B.om(C~°° (K Xm Vy),C°°(K Xm V y ))). Restricting the smooth distribution 
kernel to the diagonal, multiplying by the smooth compactly supported function q, and inte- 
grating over ia* we obtain the following estimates using the standard theory of the Euclidean 
Fourier transform. For any JVsN 



sup sup | / Q(X,a)(k,k)q(X)dX\a 2p (l + |log(a)|) 

k£K aeA+ Jia* 



N <oc 



keKaeA+ Jia 

sup sup 

k&K a&A + 



| / X (kfb{w)T o $ 0i _ A o J- A o (1 - x) o (ext o tt* - 1) o $ A , o i?*£lL^](fc, k)q(X) a - 2X d\\ 

Jia* Ca-{A) 

(l + |log(a)|) iV <oo 



sup sup 

k&K a&A + 



I / X(kf[^-T o $ 0>A o (ext o 7T, - 1) o (1 - x) o (J£_ A )* o $_ A , o i?*7(«;)*](fc, A:)g(A)a 2A dA| 
(l + |log(a)|) JV <oo. 

This implies the proposition. □ 



Remark : We have shown in fact that q i-> |(-A g )|dj og | is a continuous map from C^°(ia*) to 
L 1 (r\G). It would be desirable to extend this map from C^°(ia*) to the Schwartz space S(ia*). 
It is this technical problem that prevents us to prove that the Fouriertransform $ of * restricted 
to the unitary principal series representations is a tempered distribution. If this would true then 
it is in fact a measure and given by our computations below. 

If we would like to show that the map q t— > |(Aj)|diagl extends to a map from the Schwartz 
space to L 1 (r\G ! ) along the lines above we need estimates on the growth of ext as the parameter 
A tends to infinity along the imaginary axis. If the imaginary axis is in the domain of convergence 
of ext, i.e. the critical exponent 5r of T is negative, then such an estimate is easy to obtain. In 
the general case one has to estimate the meromorphic continuation of ext, and this is an open 
problem. 



4.4 A computation 

In this subsection we want to express J r \ G tr A q (g, g)no(dg) in terms of q. 

Recall that the symmetric space X = G/K can be compactified by adjoining the boundary 
dX = G/P. As a convex-cocompact group T acts freely and properly on AUO with compact 
quotient. Therefore, we can choose a smooth function x r € C^°(Xu£l) such that X^er 1*X T = 1 
on lull. The restriction of \ T to X can be lifted to G as a right- K-invariant function which 
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we still denote by x • We denote by Xoo the right-M-invariant lift to K of the restriction of x 
to dX = K/M. We write 

/ teA q (g,g)u. G (dg)= / x r (#)tr A q (g, g)fi G (dg) . 
Jr\G iG 

Let xt/ be the characteristic function of the ball By in X of radius f/ centered at the origin [K]. 
Again, we denote its right-if-invariant lift to G by the same symbol. Then we can write 

/ x r (g)to A q (g,g)u, G (dg) = lim / x T (g)xu(g)^ A q (g, g)^ G (dg) . 
Jg u ^°° Jg 

Given U we fix a function xi £ C^°(G/K) such that 

XiX(7X r = X(7X r • (14) 

The operator Xc/X 1 " -^ ( ex * ° n * ~ 1) ° (°^ >R a)*Xi has a an integral kernel of compact support. 
Since the kernel is smooth in the interior of the support it is of trace class. We can write 

/ x r (g)xu(g)trA q (g,g)u. G (dg)= [ Tr [xuX^Px ° M o vr* - 1) o (°P R x T X i]qWdX . 
Jg Jia* 

Note that 

ia * 3 A _> Tr [xuX^P^ ° (ear* o tt, - 1) o (°P fi A )*Xi] 

is a smooth function. We want to compute its limit in the sense of distributions as U — ► oo 
using Green's formula. 

Note that Vb(1 p+q ) is a complex bundle with a real structure which is trivial together with 
this structure. Indeed, B is orientable, and Vb(1 p+q ) is a real power of A max T*i?. We choose 
any non-vanishing positive section 4> £ C°°(B, Vb(1 p+q; )). For any z G C we can form c/> 2 G 
C°°(B, Vb(1 p+2Q ,)). In particular, if we choose z such that za = A — fi, then multiplication by <^> z 
gives an isomorphism $ A/i : C°°(B, Vb(<t m )) — > C co (B, Vb(<J\)), and similar isomorphisms of the 
spaces of L 2 - and distribution sections. If Re(z) = 0, then ext : C°°(B, V B {lp +za )) ^i'^"*" 
is regular (indeed p + za belongs to the domain of convergence). Multiplication by ext(4> z ) gives 
a continuous map 

A >M oc — oc • 

This map is T-equivariant and extends in fact to larger subspaces of HZ'^ of distributions which 
are smooth on neighbourhoods of the limit set A. 

The ususal trick to bring in Green's formula is to write 

Tr [xuX F0 Px ° (ext o Tr* - 1) o (°P R x Txi] 

= lim Tr [ X uX r °Pl ° (ext o ^ o vr* - § A;/1 ) o (°P^)*Xi] • 

— * A 
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Let V 7 denote the invariant connection of the bundle V(7) = G X K ^7 over X and A 7 := 
— (V 7 )* V 7 be the Laplace operator. Then there exists a constant c £ 1 such that (A 7 + c + 
A 2 ) o pT = 0. Let n denote the outer unit-normal vector field at Bjj. 
By Green's formula we have for A 7^ ±/x 

Tr [XUX F °PI o (ext o $ A> „ o vr* - l» A , M ) o 

= -jTT^ Tr I A 7> X r ]Xf/ o °P A T o (ex* o $ AjM o Tr* - o (°P^)*xi (15) 

" A^2 Tr ^ ° ^ W ° ° $A, M o Tr* - § AjM ) o (V 7 °P4)f 8Su (16) 

+ >^2 Tr ^ i^Plhsu o (ext o ^ o vr, - § AjM ) o (°P^ 9Bu . (17) 

Note that the derivatives of xi drop out because of (14). Moreover, (°P^)\g Bu : — ► 
C°°(dBu, V^iQBy) denotes the composition of the Poisson transform and restriction to the 
boundary of Bjj, and this operator can be expressed in terms of VT „■ 

We introduce the following notation. Let ajj £ A + be such that distx (ajjK, K) = U. We 
define ui(U) := a~[j 2p vo\{dBu)- Note that uj := lim[/_» o0 u(U) exists. Let Xa £ C°°(K) denote 
the function k 1— > \ l \ka). Note that lim^oo \a = Xoo- Then we can write 



(ED + I 

+^f^2 Tr & dv Lu ° ( ext ° ^ - *a. m ) ° cp-^)* > 

Here dV\ a stands for the derivative of the function a \—* V\ a with respect to the positive 
fundamental unit vector field on A. 

Let x be a smooth cut-off function on X U £1 of compact support such that 

7 supp(x) n supp(x) = , V7 € T , 7 ^ 1 . (18) 



Note that x r can be decomposed into a finite sum x r = Yli X % such that each x % satisfies (|l~8|). 
We fix a cut-off function x on f2 satisfying ( |l~8| ) and x = 1 on a neighbourhood of supp(x) H f&. 
We further define Xa(^) = x(ko-K), Xoo(k) = X|dx(^-^0 an d observe that |xa — Xoo I = 0(cT a ) 
for any seminorm |.| of C°°{K). Using that x (ext o $ A)/Lt o 7r* — $a„u) X = we can write 

(16) + @ (x r replaced by X ) 

= ~W^ Tr Xau v ^ o( - 1 -^° ( ext ° ° n * - ^ (W-wuY 

,Tr Xot , o V T Kau o x o (ext o $ Aj/1 o tp* - * A ,„) o (1 - *) o {ffP*^)* 
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2p 
2p 

+^^2 Tr ° ° X ° (ear* o § A>/i o tt* - $ A)/1 ) o (1 - x ) o (7^)* . (19) 

We now insert the asymptotic decomposition (R) of the operators as a — ► oo noting 
that in each line one of these operators is localized off-diagonally. In order to stay in trace 
class operators we choose a function xi £ C°°(K) such that supp(l — xi) H supp(x) = and 
supp(l — x) H supp(xi) = 0- We obtain 

(16) + (0) (x r replaced by X ) 

= - " Ua \ 2 X }~2~ P) T * ^^=^7^)TXoo$o,-a<a(1 " x) (ext o $ A> „ o vr, - $ A , M ) 

W£ ,a& V " P) Tr ^ rXoo$0 _ A j- a(1 _ ^ ^ ^ _ ^ (JF^)*$_ Mj0Xi 

Tr i?* 7 ( u; )-l-^Lrxoc*o,AX (ext o «§ A , M o vr* - 5 AjM ) (1 - X )( J^)*<I>_ M ,oXi 
Tr R*T X oo$o,x < A X (ext o ^ o Tr* - <I A , M ) (1 - x)(^-^)**-m,oXi 



A 2 


-M 2 






A (p- 


p) 


A 2 


-P 2 






A (p- 


p) 



A 2 -/x 2 

+ ^ a ^ 2 _ ( ^ 2 A ~ P) Tr i?*^^! 7 ( u; )r Xt30 $o > -A^A(l " x) o ^ o vr, - $ A , M ) ^ oXl 
+ a^_(-A-p) ^ i? * TXoo#o _ a j- a(1 _ x) ( ext o ^ o ^ _ $ A)M ) (J-_^)*$_ M)0X1 

+ UJUa f 2 X[X ~ P) Tr i?* 7 («;)-i-^T Xoo $ Q)A (ext o ^ o ^ - $ A)/U ) (1 - x)(^- m )**-m,oX1 

fi— A/ > \ 

+ ^ X 2_~ 2 Tr ^*r Xoo * ,-A^A X (ext o $ A> „ o Tr, - <l AiM ) (1 - x)(Jff,- M )**-M,0Xl 
+a U a ^~2 R x(^ 

= " T+l" ^ ^^I^7(^)rxoo^o,-A< A (1 - X) (ext o * A> „ o tt, - $ AjM ) <D Mi0 xi 

/i— A 

- T^ Tr #*T X oc$o,-A < A (1 - x) [ext o $ A(/i o Tr* - $ Ai(U ) J^S-^oXi 

A — /i 

+ ^-Tr i ? * 7 ( w )-i^Ir Xoo ci>o, A (ext o § A , M o tt, - * A ,„) (1 - xJC^-^^-^oXi 

A + /I C<7^ — AJ 

/x— A 

_a^a^ Tr i? * TXoo$o _ a j^ aX $ ^ _ $ A ) (i _ ^(^-/J^-oXl 
The remainder R X (X, fi,a) is holomorphic and remains bounded in C°°(io* x ia*) as a — > oo. 
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We define (R,T) G C such that R* oT = (R,T)id Va . If a is not Weyl-invariant, i.e. a w ^ cr, 
then the compositions R*c^(—[i)* , y(w)T, R*j(w)~ 1 c~ f (\)T vanish. If the representation a is 
Weyl-invariant, then it can be extended to the normalizer Nk(M) of M. In particular, we can 
define a(w). In this case we define (R,T)(\) £ C such that 

CJ Hii* 7 H- 1 ^^ T r = (R,T)(\)id Va . 

c CT (-A) 

Note that R* ^^ ^(w)Ta(w)~ 1 = (R,T)(-fi)id Va . Further we put J CTjA = a(w)J™ x : £££ -► 
H^^. Then we can write 

Tr[ X Xu o °P A T o (ext o $ A) ^ o tt* - * A)M ) o (°P? M )*xi] 

= - Uua ^ ^ r )(~^) Tr $ Xoo j o (1 - x) o (ext o $ A o Tr* - $ AiAt ) o Xl (20) 

+ ^^ A ^ T)(A) Tr Xoo o (ext o $ A ^ o 7r* - $ A)M ) o (1 - x) o (./*,_„)* o o ^21) 

+ Tr ' t A 7' X] ° {ext o ^ o vr, - $ A)M ) o ( P« M )* (22) 
+ _ Jp Qxi^ M> a u) 

Q x (\,H,au) (23) 
:= R x (X,fi,au) 

-a^Tr' [A 7 , X ](l - Xc/ ) o °P A T o (ext o $ A> „ o tt* - $ Ai/1 ) o (°P^)* (24) 



We defer the justification of the terms (p2|), (p4|) to Lemma 4.3 below. The functional Tr' here is 
applied to operators with distribution kernels which are continuous on the diagonal, and it takes 
the integral of its local trace over the diagonal. Note that the remainder Q x is independent of 
the choice of Xi- 

The left-hand side of this formula is holomorphic on x a£. The terms on the right-hand 
side may have poles. To aim of the following discussion is to understand these singularities 
properly. 



Lemma 4.2 

1 



-Tr Xoo o J^ A o (ext o $ AijU o vr* - $ A)/X ) o ( Jf _^)* o $_ Mj _ A o X i 



A — 

is regular for fx = A . 
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Proof. We must show that 

Tr Xoo o J^ A o (ext o tt* - 1) o (J^ x )* o Xl = . 

Recall the definition of the scattering matrix ST\ from ||, Def. 5.6. We are going to employ 
the relations 

ext o S™ x = < A o ext , vr, o (JJ»_ A )* = (^_ A )* o vr* , S™ x o (5^_ A )* = id . 

We now compute 

Tr Xoo o J™ A o (ext o Tr* - 1) o ( J^_ A )* o Xl 

= Tr ( Xoo o ext o S£ A o (^_ A )* o vr* o Xl ~ Xoo ° < A o ( Jf^)* o X i 
= Tr Xoo ° (ext o 7r* - 1) o Xl 
= . 



□ 



In particular we have 



u(U)q£- x (R ,T) 

U^oo fi^X A — fl 



lim lim - 1 ; / Tr ( ext §A ^ ^ _ ^ (J ™ $ _^_ A X1 



= ljx(R,T)^- Tr X o J^ A o(exto$ Ai/ ,OTr*-$ Ai/i )o(J|; )*o$_ M) _ A o Xl . (25) 
"A* |/Lt=A 

If the distribution kernel of an operator admits a continuous restriction to the diagonal, then 
let T>A denote this restriction. 

Lemma 4.3 1. For any compact subset Q C ia* there is a constant C such that for all k £ K 
and fj,, A G Q 

\V[A J: X ] o °P A T o (ext o $ Aj/1 o tt # - $ A)M ) o (°F* )*(fca)| < Ca^~ a . 

VKe have 

Tr' [A 7 , X r ] o °pj o (ext o $ Aj/1 o tt # - $ Aj „) o (°P R ;)* = 
(note t/iat we consider the cut- off function x r here). 

Proof. The reason that 1. holds true is that \d\{ka)\ < CaT a and |A X (&a)| < CaT a uniformly 
in k £ K and a £ A. We use the decomposition 

P[A 7 , x] o °P A T o (ext o $ A)/i o Tr, - 5 A)M ) o )*(fca) 

= P[A 7 , X ]o°P A T oxo(exto|» AiAt o^-$ AiM )o(l-x)o( p4)*(A:a) (26) 
+P[A 7 , x ] o °P A T o (1 - x) o (ext o $ AiM o Tr* - * A ,„) o (°P^)* (fc a ) . (27) 
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4 POISSON TRANSFORMS AND ASYMPTOTIC COMPUTATIONS 



The asymptotic behaviour (0) of the operators V\ a is uniform for A in compact subsets of ia* 
and can be differentiated with respect to a. We conclude that for any compact subset Q C ia* 
there is a constant C £ R such for all A, \i E Q we have 



sup |P[A 7 , X ]a o °P A T o (1 - x) o (ear* o § Aj/i o ^ - $ A>M ) o (°P R )*(ka)\ < CaT 2 ^ 
fc 

sup|P[A 7 ,x] a o P A r o(l-x)o(exto$ Ai/1 o7r*-$ AiM )( P4r(A ; a)| < Ca^ . 
fc 



We can write x r as a finite sum x r = ^2 i x 1 ^ where the cut-off functions \ % obeying (18). 
For each index i we choose an appropriate cut-off function x\ as above. It follows from 1. that 
tr P[A 7 , x r ] ° °Px ° {ext o i A (U o vr* - 5 A)/Lt ) o (°P? M )* is integrable over G. We compute 



Tr' [A 7 , X r ] o °P A T o [ext o $ A)/i o vr* - 5 A)M ) o (°P 

= ^ Tr' vr( 7 )- 1 X r 7r( 7 ) o [A 7 , X r ] o °P A T o (ext o £ A ^ o vr* - * A)M ) o (°P^)* 

= ^ Tr' X r o [A 7 , ft" 1 )*/] o tt( 7 ) o °Pj o (ext o $ v o vr, - $ A , M ) o Tr^)" 1 o (°P^) 
7er 

= ^Tr'x^tA,,^- 1 )^ 
7er 



x; , n / x r o[A 7> ( 7 - i )v 

7er 

^Tr' X r o[A 7 ,( 7 - 1 rx r 
7er 

^Tr' X r o[A 7 ,( 7 " 1 )*x r 

7er 





°Pf o (vr <T ' A ( 7 ) o ext o <| A;M o 7T* - ^ CT ' A ( 7 ) o $ A>M ) o ^( 7 )- x o ( U P 



°P A T o (ext o $ A>M o Tr* - $A,/u ° 7T^( 7 )) o ^(t)" 1 o (°P* )* 



°P A T o (ext o ^ o vr, - $ A) „) o 7r<n 7 ) o Tr^)" 1 o (°P* )* 



o °P A T o (ext o $ AiM o 7T* - i A)M ) o (°P^)* 



since E 7 er(7 X )*X V = 1- 

Note that the second assertion of the lemma implies 

^ Tr [A 7 , xi o °P A T o (ext o <I A>M o vr* - § A , M ) o (°P^)* o X j = 



□ 



We now combine ( p0|) and ( |2l| ) and write 
(20) + © 

■(/2,T)(-m) 



a;(C/)(a£ +A -q-^) 



A + /U 

Tf $ Ml -A ° Xoo ° Ja,X (1 - X) ( ea ^ ° $A, M o 7T* - $A„u) o Xl 



(28) 



" l< ' -Tr [(R, T)(X)xoo o (ext o § A>/ , o tt* - * A)|t ) o (1 - x) o (J*,- M )* o $_ MjA o Xl 



A + /i 



-(P, TX-^i)*^ _ A o xoo o J CTjA o (1 - x) o (ext o $ AiM o Tr* - $ A , M ) o xi] (29) 



4.4 A computation 
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Note that ( |28[) is regular at A = = 0. In fact, if ext has a pole at A = 0, then it is of first order 
and J CTj o = id (see Prop. 7.4). Hence, the composition Xoo ° ^<r,o ° (1 — X) vanishes. 
In order to see that ([H]) is regular at A = fj, = 0, too, observe in addition that 

TV [x o (ext o 7r* - 1) o (1 - x) o ( J 5j0 )* o xi - X ^<r,o ° (1 - A) (ext o 7r* - 1) o xi] = . (30) 



Combining Lemma p~2| and [4.3| , 2., and (30) we conclude that jtzt~i Q x r(A, fi, ay) is regular 
at fj, = A, where we set Q x r := ^ Q x *. Furthermore, locally uniformly in A 

aT a 

lim —r^- — ? Q r (A, /i, au) = . 
U-*oo \ A — n, z A 

By the Lemma of Riemann-Lebesgue we have 

lim 1 ( u TV [(12, T)(A)xoo o (ext o Tr* - 1) o (1 - X ) o ( J 5 ,_ A )* o $_ A , A o X i 

-{R,T)(-\)$x,-\ ° Xoo ° Ja,X (1 - X) (ext O 7T* - 1) O xi] 



as distributions on io*. Moreover, 

Tr $ A _ A o Xoo ° Ja,\ (1 - x) (ext o vr* - 1) o xi (31) 

= irtux(R,T)(0)5 (\) lim [Tr $ A ,_ A o Xoo o J ffjA o (1 - x) o (ext o 7T* - 1) o X l] (32) 

A — *0 



in the sense of distributions on ia* . Combining (J25|) and (J32j) we now have shown the following 
proposition. 

Proposition 4.4 

tr A q (g,g)n G (dg) 



u x (R, T) f J2 4- Tr [xl o J- A o (ext o l» A , M o tt # - $ AjM ) o ( J^J* o $_ M ,_ A o x \] 

J ia* ■ a l 1 \iL=\ 



+ iroj x (R, T)(0) ^ lim (Tr [$ A ,_ A o ^ o J CTiA o (ext o 7T* - 1) o X \) ) q(0) . 

i 



Observe that we can rewrite this in the more invariant form 



u x (R, T) f -j~ Tr ' [xl ° J*,x (ext o $ v o Tr, - i A(At ) o ( Jff )* o <D_ /tj _ A ] g(A)dA 

An* "M|u=A 



+ vrw x (fi,T)(0) lim (Tr' [$ A ,_ A o x^ o J CTjA o (ext o tt* - 1)]) q(0) . 

A — >0 
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5 THE FOURIER TRANSFORM OF * 



5 The Fourier transform of ^ 

5.1 The contribution of the scattering matrix 

We consider a symmetric function q G C^°(ia*, End(V r 7 (8> Honnf(V 7 , Va)))- For f £ G we form 



/ig(vr) G End(Vn-) as in Subsection There we have seen that R{h q ) and Rr(h q ) have smooth 
integral kernels. 

We choose a basis {v i} i= i i ... ) dim( 7 ) of and a basis {Tj}j=i j ...,dim(HomM(V 7) V CT )) of Hom A f (F 7 , V a ) 
Let {v 1 } be a dual basis of and {T J } be a dual basis of Homjv/(V^, Vy) = Homif(^ 7 , V a )* 
(with respect to the pairing (T',T) = try 7 T" o T, T £ Hoitlm(V^, V a ), T' G Homjvf(V r ff , V^,)). 
Then := u j ® Tj} can be considered basis of if CT ' A (7). Furthermore, := <g> (T J )*} 
can be considered MS M basis of iT*- A (7) = H a > x and we have 

' (T J o 7 (fc)- 1 (^),(T z ro 7 (A ; )- 1 (^))dA ; 



IK 



(-f(k)oT l oT j o 1 (k)- 1 (v i ),v h )dk 



For g £ G and to G we define P\ w (g) G A such that (P^" ((f))(g),w) = (jp\ w (g) , <j>) for all 
cf) G ITZ'ao- Using (||) we can write P\ w (g) = ir cr '~ x (g)(w (g) T*). We can write 

where the functions g^fe := (u* ® T^,q(vk ® 7})) belong to C£°(za*). Now we can compute 
Tr 7^(5)^(7^) = dim(y 7 )^(^,vr CT - A ( 5 )/ lg (7r CT ' A )^) 

= dim(y 7 ) £<Pf (M^MjOGT 1 ),* 4 ) 
i,i 

= dim(y 7 ) ^(hg^fap^te- 1 )) 

= dim(y 7 ) ^2 Qklij(X)(vk®ThP\ ! tV i(g~ 1 )) 



i,j,k,l 

dim(l/ 7 ) QMij (A) (P^l (P\^ v i (g~ 1 ) ) ( 1 ) , f fc ) 

i,j,k,l 

dim(y 7 ) ^ ^(A)(^,Pf o (P^rGT 1 ,!)^)) 



i,j,k,l 
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In the last line of this computation P^ 3 o (P_\)*(g, g') is the integral kernel of the G-equivariant 
operator 

PF (P-iT ■ C-°°{G V 7 ) - C°°(G Vj) . 
We can express the integral kernel of R(h q ) via Poisson transforms as follows: 

K R(h q )(9,9i) = j\ Trvr( 5 )/ig(7r)7r(^ 1 )p(^) 

= TrTr^M^'VWdA 



= E JM y ,,) / «(A){»-,i'ro(P_ T {)-( 5 - 1 91 ,l)fe))P.(A)«iA 
_ E dh^)dim(V 7 ) / ^y.jf (j5)-tel,»)ta))lv(A)« 



i,j,k,i 

In a similar manner we obtain 



Tr -K a ' x {g)h q {-K a ' x )-K a ' x (g^ 1 )ext vr* 

= Tr ^(tt^K'V -1 )^ ^* ^G?) 

= dim(y 7 ) ^(Tr^^^^^r^ext Tr* ^ X (g)<p k i) 



= dim(V 7 ) ^(^(Tr' 7 ^)*^,^^^- 1 )^ Tr* ^ A (<7)0«) 
= dim(V 7 ) £ ^(A)(^ ® ,7r cr ' x (g^ 1 )ext vr* ^ A (<7)<fe> 
= dim(V 7 ) ^ qkUj(X)(v\Pr(extTr*ir a > X (g)<p kl )( gi )) 

i,j,k,l 

= dim(y 7 ) ^ qklijW{ext^Ti^ x {g)4) kh pl^ vl {gi)) 

i,j,k,l 

= dim<y 7 ) qkiijW{<i>khK a - x {g- l )extit*{pT 3 vl {gi))) 

i,j,k,l 

= dim(y 7 ) ^2 QkiijW(vk, P^ l x (ext 7^(^(51 )))(#)) 

i,j,k,l 

= dim(y 7 ) ^ q kU j{\){v\Pl 3 oexto^ o (P^ l x )*(g 1 ,g)(v k )) 

i,j,k,l 



and thus 
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5 THE FOURIER TRANSFORM OF * 



i,j,k,l 

We conclude that 



E dim( ^ )dim(I/7) / ftwCAXt;',^ o ester, o (P T $)*(5i,<7)(^))p CT (A)dA 

^ dim(V;) dim(U 7 ) f 4 T , ^o d t m*/ \/ ujvq^ 

> / qklijW{v , P\ oexto7r^o( P_ l x ) (gi,g)(v k ))d\36) 

<—f 47TUJX Jia* 



K 



R T ( hq )(9,9i) ~ K R[hq) (g,gi] 



i,j,k,l 



dim(U CT ) dim(U 7 ) ^ 4 • 

4^ E (« ,^W^,I} )(Si,sH) 

i,j,k,l 



By Proposition |4.1| the difference 

5 ^ ( K R r (h q )(9,g) - K k(hq) {g,g)) 
is integrable over Using the fact that 7 is irreducible we compute 

i,k Jr \ G Jk 

u Mg 



The following formula is now an immediate consequence of Proposition 4.4 



| ^ (^ r (M 5) - ^ (/lg) (5, 5)) ^Gidg) (37) 
dim(U a ) dim(U 7 ) 



4tt 



/ EE( T *> Ti ) 



i k,j,l 



-f- Tr [xio o < A ° (eat o $ A , M o 7T* - $ v ) o (J| | )* o <D_ Mj _ A o x |] g Wfci (A)dA 
a f i \ f i=X 

+ dim(K) 4 dim(n) EE^,^)(0) lim (Tr [$ ArA oi°^oM^«-l)oxi])tei(0) . 

Now we will rewrite this formula in a more invariant fashion. Using ( |34|) we first compute 

E<^ ri >«M*j(A) (38) 
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-Tr/j^vr^) (39) 



1 



dim(F 7 ) dim(K) 

We assume for a moment that a is Weyl-invariant. For T E Hom M (K, F 7 ) let T" £ Hom(if^, F 7 ) 
be given by = Pj(/)(1). Recall the relation (@, Lemma 5.5, 1.) 

T tt o a(^) o J™ A = Cff (-A)[ 7 (u;) o c 7 (A) o T o ^(uj)- 1 ]* . 

We compute 

(B ' r>(0) = dnTOo) TVjW ° g ° 7W " 1 ° c ' (Q) ° T 

Tr 7(10) o c 7 (0) o T o cr(u;) _1 o jR* 



dim(K)c CT (0)^ J M ' 
1 



dim(K)c (T (0) ^ 



Tr j(k) 1 o7(w)oc 7 (0)oTo(j(w) 1 o R* o j(k)n, K (dk) 



dim(K CT )c CT (0) "/ft: 

1 V^/nTH oc 7( )°' ro ' 7 (' u )~ 1 



dim(y CT ) C(J (0) 

= dim(K) CfT (0) ^ ([7W ° C ^ (0) ° T ° ® 

= di^ryE^^®^)^^ 71 *) 

Since J CTj o is -fT-equivariant, we can write 

J ff ,o(vi ® R) =Vi® j a ,o(R) 
for some j a> o G End(HomM Vy)). We now have 

= ^tttE^®^'^)^®^^))) ( 4 °) 
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5 THE FOURIER TRANSFORM OF * 



di^T) X>* T *' qW ° JCT ' o(wfe T '* }) 

Trh q (TT°>°)oJ afi (41) 



k,I 

1 



dim(T4) dim(l/ 7 ) 

Inserting (^) and ( f4~I| ) into ( p7| ) we obtain the following theorem. 

Theorem 5.1 If q is a smooth compactly supported symmetric function on M x ia* such that 
q(a, A) £ End(Ky (g) HomM(V^, V^)), t/ien the difference 

9 >-> K R r [h q )(9,g) ~ K k{hq) {g,g) 
is integrable over T\G, and we have 
c g ( K R r (h q )(9,g) ~ K Rihq) {g,g)) H G {dg) 



4-7T 

Tr [xl o J- A o (ext o ^ o vr* - * AjM ) o (Jg )♦ o $_^_ A o x »] Tr h q (^ X )d\ 

a l J '\ti=X 

+ 7 ^ lim (Tr [$ A - A ° X*oo ° ^<x,A ° (ext o 7T* - 1) o x \] ) Tr h q (-n ufi ) o J CTj0 . 

i 

We can again rewrite this formula as follows 
/ ( K R r (h q )(9,g) ~ K R(hq) (g,g)^j n G (dg) 

= E — 

Tr' o J- A o (ext o 3> A , M o vr* - * A| „) o (J| ; )* o d»_^_ A ] Tr ^(7r CT ' A )dA 

"M |/Lt=A 

+ - lim (Tr' [$ A ,_ A o x ^ o J CTi0 o (ext o Tr* - 1)] ) Tr h q (ir°>°) o J CTi0 . 
5.2 The Fourier transform of A conjecture. 



Observe that Theorem 5.1 does not solve our initial problem of computing the Fourier trans- 
form of the distribution ^. The point is that there is no function / G C£°(G) such that 
its Fourier transform / has compact support. In order to extend Theorem |5.1| to / we must 



extend Proposition 4.1 to Schwartz functions. As explained in the remark at the end of Sub- 



section the main obstacle to do this is an estimate of the growth of the extension map 
ext = ext A : C~°°(B, Vb(o~, A)) — > as A tends to infinity along the imaginary axis. 



5.2 The Fourier transform of Vf. A conjecture. 
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The goal of the present subsection is to rewrite the result of the computation of ^f' in terms 
of characters thus obtaining the candidate of the measure We will also take the discrete 
spectrum of L 2 (T\G) into account. 

Recall that 

Tr/V' A ) = *W(/) • 

If 7r cr '° is reducible, then it decomposes into a sum of 7r CT,+ © ir u ~ of limits of discrete series 
representations which are just the ±1 eigenspaces of J^o- In this case ext is regular at A = 
(||, Prop. 7.4.). We can write 

Tr/V'°) o ^,0 = ^,+ (/)-^-(/) • 



If we replace h q by / then formulas (^5() and (|3q ) just give the contributions of the continuous 
spectrum K Rac ^ and K R ac^ to the integral kernels K R ^ and K Rv ^y We have K R ^ = 
K R ac (f) + K Rd{f) , where 

K Rd(f)(g,9i) = Tr7T (9) filial 1 ) ■ 

7reG d 

Since we assume that / is iT-finite, and there are only finitely many discrete series representations 
containing a given i^T-type, this sum is finite. 

Furthermore, K Rr(f) = K R «c (f) + K R d {f) + K R c {f) . Here K R d (f) = EneG d K n^(f) is the 
contribution of discrete series representations (again a finite sum), 

K R d,n {f) (g,gi) = J2(i{ji,(f) j )(4}i,iT(g)f(iT)iT(gi 1 )(t> j ) , 
hi 

where {4>j} and {tpi} are orthonormal bases of the infinite-dimensional Hilbert spaces V n and 
M n , respectively. The finite sum K R c^ = Yl^eC! ^R^U) 1S ^ e discrete contribution of repre- 
sentations belonging to G c . If we choose orthogonal bases {<fij} and {ipi} of the Hilbert spaces 
V n and M n (note that dim(M 7r ) < oo), then we can write 

K R^(f)(9,9i) = J2(^i,<Pj)(^i,^(9)f(^(9l 1 )<Pj) ■ 

We define the multiplicity of ir by Ny(tt) := dim(M 7r ). It is clear that 

K RC{f) {g,g)^ G (dg) = ^ JV r M<W) • 

7TfcCr c 

In Lemma 6. 13| we will show that if / is iT-finite and invariant under conjugation by K, then 
for each ir £ G^ we have 

K R d^ {f) (g,g) - K Rd ,n {f) (g,g) G L x (r\G) . 
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5 THE FOURIER TRANSFORM OF * 



Given any A £ V# # (8> V^jf we define the function G 9 7r' ^ /^(vr') to be zero for all ir' ^ it 
and Ha{^) '■= A, where A := J K n(k) (g> ir(k)Adk. The map 

Vfr,K ® Vmt 3 ^4 ■-> := J. K R*(h A )(9,9) - K kd{hA) {g,g)]^ G (dg) 

is well-defined and a (g, ivT)-invariant functional on V^.^ 8) V^RT- Since is irreducible it 

follows that 

T{A) = N t {-k)0«{A) 

for some number iVr(7r) G C which plays the role of the multiplicity of n. Here we consider A 
as a finite-dimensional operator on V n . 

It follows from Lemma 5.13 that Kpac^{g,g) — Kf> ac ^(g, g) belongs to L 1 (Y\G) not only 



if / is smooth of compact support, .fT-finite and i<C-invariant (Proposition |4,1| ), but also in the 
case that / £ C^°(G) is K-finite and i-T-conjugation invariant. 

The following conjecture provides the candidate for the measure <3?. Its discrete part is 
expressed in terms of multiplicities Ny(tt). If tt £ G c , then Nt(tt) is just the dimension of the 
space of multiplicities M n and thus a non-negative integer. If 7r £ G^, then A r r(vr) is a sort 



of regularized dimension of M n . We will show in Proposition 6.14 that Nt(tv) is an integer in 
this case, too. The continuous part of the spectrum will contribute a point measure supported 
on the irreducible constitutents of the representations 7r cr '°, and the corresponding weight will 
be denoted by Ny(tt). The remaining contribution of the continuous spectrum is absolute 
continuous to the Lebesgue measure dX on G ac = M x ia* + and will be described by the density 
Lr(vr CT,A ). By Theorem 6.12 this density appears in the functional equation of the Selberg zeta 
function. In particular, as it can be already seen from the definition below, Lr(7r CT ' A ) admits a 
meromorphic continuation to all of as a function of A. Its residues are closely related to the 
multiplicities of resonances. 

Conjecture 5.2 If f £ C^°(G) is bi-K -finite, then we have 

*(/) = E J" / L T (^ X )0^Af)d\ 

+ E E M^ e )o^u) 

cr£M,o-^ia w ,7r CT '° irred. 
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where 

Lr(vr<J ' A) := 77, Tr ' \&> ° J ^ ° ( ext ° ^ ° " ° (J ^* ° $ ^>- A ] 

and 

iVrK^) := ±^Tr' Urn ([$ A ,_ A o x ^ o J CTjA o (ext o vr* - 1)]) 

iV r (^°) := ^Tr' lim ( [cD Aj _ A o x ^ o J CTjA o (ext o tt, - 1)] ) . 

Note that the discussion above does not prove this conjecture. What it does prove is the 
following theorem. 

Theorem 5.3 If G 3 tt h(ir) 6 EndK(V w ) is smooth, of compact support, and factorizes over 
finitely many K -types, then we have 

y'(k) = El - / L T {Tr a ' X )Tr h(TT a ' X )d\ 

+ E E Nr{«"*)T± W) 

aeM ,a=a w ,n a '°red. 

+ Yl iVr(7r CT ' )Tr^(7r CT ' ) 

a£M,a=a w ,7r CT -° irred. 

+ ^ ^r(vr)Tr h(ir) . 

7reG c UG d 

6 Resolvent kernels and Selberg zeta functions 
6.1 Meromorphic continuation of resovent kernels 

We fix any K-type 7. The Casimir operator 17 of G gives rise to an unbounded selfadjoint 
operator on the Hilbert space of sections I?{X, 1/(7)). To be precise it is the unique selfadjoint 
extension of the restriction of fi to the space of smooth sections with compact support, where 
we normalize £1 such that it is bounded from below. For each complex number z which is not 
contained in the spectrum cx(fi) we let R(z) be the operator (z — fi) -1 . By r(z) we denote its 
distribution kernel. 

The Casimir operator descends to an operator acting on sections of Vyipi) and induces an 
unique unbounded selfadjoint operator Qy on L 2 (Y,Vy(~f)). For z a(Qy) we define the 
resolvent Ry(z) of the operator (z — J7y) _1 and denote by ry(z) its distribution kernel. 

We consider both, r(z) and ry(z), as distribution sections of the bundle V(7) V(-y) over 
X x X. We will in particular be interested in the difference d(z) := ry(z) — r(z). 
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6 RESOLVENT KERNELS AND SELBERG ZETA FUNCTIONS 



Let M(7) C M denote the set of irreducible representations of M which appear in the 
restriction of 7 to M. For each a G M and A G aj. let z CT (A) denote the value of on the 
principal series representation H a ' X . Note that z a (X) is of the form z a (X) = c a — A 2 for some 
Co- G K. We define C 7 to be the branched cover of C to which the inverse functions 

X a (z) = yjc a - z 

extend holomorphically for all a G M(pj). We fix one sheet <C phys of C 7 over the set C \ [b, 00), 
6 := min ^ c CT , which we call physical. We will often consider <C phys as a subset of C. It follows 
from the Plancherel theorem for L 2 (X,V{^)) and L 2 (Y, VV(t)) that [6,00) is the continuous 
spectrum of and Oy . Thus is defined on the complement of finitely many points of C phys 
which belong to the discrete spectrum of and f2y. 

Let A denote the diagonal in X x X and define S : =Ui^7er(l x 7)^ C X x X . Let f2j, 
i = 1,2, denote the Casimir operators of G acting on the first and the second variable of the 
product X x X. The distribution d(z) satisfies the elliptic differential equation 

(2z - fii - n 2 )d(z) = (42) 
on X x X \ S and is therefore smooth on this set. 

Lemma 6.1 d(z) extends to C 7 as a meromorphic family of smooth sections 0/1/(7) ® ^(7) 
onIxI\5. 

Proof. We first show 

Lemma 6.2 r(z) and ry(z) extend to C 7 as meromorphic families of distributions. 

Proof. We give the argument for ry{z) since r(z) can be considered as a special case where T 
is trivial. Let C X, i = 1, 2 be open subsets such that the restriction to V{ of the projection 
X — > y is a diffeomorphism. We consider </> G C£°(Vi, V(7)), Y> G C,? ^, ^(7)) as compactly 
supported sections over Y. 

We now employ the Plancherel theorem for L 2 (Y, Vy(j)) in order to show that 

r<t>,rp{z) := r Y (z)((t) <g> ip) = (<j), (z - Qy)~ V) 

extends meromorphically to C 7 . We decompose 4> = J2 s ea p (n Y ) 4>s + 4>ac, ip = J2 s ea p (n Y ) ^s + Vw 
according to the discrete and continuous spectrum of fiy. We have r ( f,^(z) = ^2 secTp ^Q Y -- > (z — 
s)~ 1 ((j) s ,'ip s ) + ((j) ac ,(z — Q)^ 1 -^^). We now employ the Eisenstein Fourier transformation in 
order to rewrite the last term of this equation. 



6. 1 Meromorphic continuation of resovent kernels 
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For each a G M("f), we consider the normalized Eisenstein series as a meromorphic family 
of maps 

C-°°(B, VbM) ® Hom M (y CT , F 7 ) 3 f ® T ^ °^(/) := °P A T o ext(f) G C°°(Y, Fy( 7 )) • 

For each a let {Tj(cx)}j, Tj(er) G Hom(V^, V^), be a base, and let T 3 (a) G Hom(V^ , V^,) be the dual 
base such that Ti(a)*T 3 (o) = 5j. If G C^°(Y,Vy(^)), then its Eisenstein Fourier transform 
EFT a (<t>)(X) G C°°(5, F B (<7 A )) ® Hom A /(^, F 7 ) is given by 

(EFT$((f))(X), f ® Tj(o-)) := (0, £ T f >(/)> = ((^f >)*(^ /> . 

As a consequence of the Plancherel theorem we obtain for 4> G C^°(Vx,V(j)),i(} G C£°(V2, ^(7)) 
that 

(<Pac^ac) = E T~~ — / (EFT^)(-X),EFT a (i;)(X))dX 



CT 6M( 7 ) ' J 



<reM(7) 

In a similar manner we obtain 

(0oc,(2-nrV« c )= E 7^— E / (^-^(A))- 1 ^, ^^^ ^™)*^))^. (43) 

We further investigate the summands in (|43| ) for each cr seperately. So for z G C phys we put 

/ (z - z^A))- 1 E^X^ °(^i (CT) )*W)rfA . (44) 

Jia* j 



u(z :-- 



If we define U(X) := u(z (7 (X)), then it is defined for Re(A) > 0. We claim that 



U(-X) := 17(A) -yE^X J ^°(fiTrW) 
3 

provides a meromorphic continuation of J7 to all of a^. Indeed, U (A) is meromorphic for Re(A) < 
0, too. We let A = e + i\i and show that the jump U{e + ifj,) — U(—e + ifi) vanishes as e — > 0. 
Using the functional equation of the Eisenstein series and unitarity of the scattering matrix we 
see that the integrand in (^J) is a symmetric function. We obtain 

lim[u(z CT (e + ifi)) - u(z a (-e + ifi))] 
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= Hm [°°[ — I — — I — — iv^^^o ^'^)*^))^^ 

Jo -e 2 - 2iefi + n 2 - v -e 2 + 2ie/i + /j, 2 - u S W» K -*fi>> JO 

= Hm f°°[ 1 1 ]y { ^0 E T^) Q (E THa) rm l dj/ 

= i im r [ — \ 1 — ]^,o^w )*w) , 1 a 

It follows from this formula that U is continuous at the imaginary axis. This finishes the proof 
of the claim. We conclude that (^) extends meromorphically to C 7 . 

Thus we have shown that ry(z) is a meromorphic family of distributions if we put the topol- 
ogy induced by the evaluations against sections of the form <f> ® rp. We now argue that it is 
indeed meromorphic with respect to the strong topology. It is clear that ry(z) is so on the 
sheet <C phys . If #1,22 £ C 7 project to z G C phys , then by the construction above the difference 
ry{z\) — ry(z2) is a meromorphic family of of smooth sections. We conclude that ry(z) is a 
meromorphic family of distributions on C~. □ 

We see that d(z) extends to a meromorphic family of distributions on C~. Since it fulfills 
the differential equation (f42"|) on A \ S we conclude that its restriction to this set is in fact a 
meromorphic family of smooth sections. This finishes the proof of Lemma |6.1| . □ 

6.2 Finite propagation speed estimates 

On X x X we define the function d {x,y) := inf 1? A 7g r d(x, 72/). Note that S = {do = 0}. Given 
e > we define the neighbourhood S e := {do < e} of S. Let b := inf cr(Oy). 



Lemma 6.3 Given e > and a compact subset W C {Re(z) < b} there is a constant C > 
such that \d(z)(x,y)\ < <7 e -(* V^M*) /or all (x,y) g S 2e and zeW. 



6.2 Finite propagation speed estimates 
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Proof. The proof is based on the finite propagation speed of the wave operators cos(t^4), cos(tAy) 
where A := \/Q — b, Ay '■= y/ily — b. We write 

rco roo 

R{z)= e~ tV ^cos(tA)dt,R Y (z) = / e - l ^~ z cos(tA Y )dt. 
Jo Jo 

Finite propagation speed gives 

/•oo 

d(z)(x,y) = / e~ tV ^[cos(tA Y ) - cos(tA)]dt (x,y) 

Jd (x,y)-e 

on the level of distribution kernels. By partial integration 

POO 

(fil - 6)^(^2 - b) N d(z)(x, y) = (b- z) 2N I e- tV ^~ z [cos{tA Y ) - cos(tA)]dt(x, y). 

Jd (x,y)~t 

We now employ the fact that f*£/ x y ^_ e e~ ts ^ h ~ z cos(tAy)dt is a bounded operator on L 2 (Y, VV(t)) 
with norm bounded by Cie~^ d °^ x ' y ^~ e ^ e ^ b ~ z ^ . A similar estimate holds for the other term. If 
we choose N > dim(X)/4, then we can conclude that 

\d(z)(x,y)\ < Ce-W^'f)"^^), 

where C depends on W, C\, and a uniform estimate of norms of delta distributions as functionals 
on the Sobolev spaces W 2N > 2 (X, V(r/)) and W 2N > 2 (Y, V Y (7)) which hold because X,Y as well 
as the bundles V (7), 1^(7) have bounded geometry. We further have employed the fact (which 
is again a consequence of bounded geometry) that we can use powers of the operator f2, £ly m 
order to define the norm of the Sobolev spaces. The assertion of the lemma now follows from 



Re( v / 6 - z) > v / 6-Re(z). □ 

The distribution r(z) is smooth outside the diagonal A because it satisfies a differential 
equation similar to (fl^). For e > we define the neighbourhood A e := {d < e} of A. 

Lemma 6.4 For e > and a compact subset W C {Re(z) < b} there is a constant and C > 
such that \r(z)(x,y)\ < Ce~^ x ' y ^~ e W b -M^) f or all (x,y) A 2e and z£W. 



Proof. The proof is similar to that of Lemma 6.2. Using finite propagation speed we can write 



r{z)(x,y)= e' 1 ^ cos(tA)dt(x,y), 

Jd(x,y)-e 



(•CO 

(fii - 6)^(^2 - b) N r(z)(x, y) = {b- z) 2N / e -*W-* C os(tA)dt(x, y). 

Jd{x,y)-e 



We now argue as in the proof of Lemma 3.3 in order to conclude the estimate. □ 



Let L 7 denote the action of 7 G T on sections of V(7). 
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Lemma 6.5 // yjb — Re(z) > 5r + p, then on X x X \ S we have d(z) = ^i^ 7 er(l ® L-y)r(z). 



Proof. It follows from Lemma U that |(l®L 7 )r(s)(x,y)| < Ce~ d{ - X ^ W b " Re W. Therefore the 
sum converges locally uniformly on XxX\S. The distribution u(z) := ry(z) — X^er^®^) 7 *^) 
satisfies the differential equations 

- fii)u(z) = 0, (z - VL 2 )u(z) = 0, 

and is therefore a smooth section depending meromorphically on z G <C phys . We further have 
the estimate Xa^yer 1(1 ® -^7) r ( z )( x > J/) I < Ce~ rd °^ x ' y \ where r < y/b — Re(z) — 5^ — p. For 
Re(z) <C we see that u(z) defines a bounded operator on L 2 (Y, Vy{^)) and therefore vanishes. 
Since u is meromorphic in z it vanishes for all z with — Re(z) > <5r + P- This proves the 
lemma. □ 



6.3 Boundary values 

The meromorphic family of eigenfunctions d(z) on X x X \ S has meromorphic families of 
hyperfunction boundary values. Since we consider a product of rank one spaces it is easy to 
determine the leading exponents of a joint eigenfunction of Oi, O2 with eigenvalue z. These 
exponents are pairs of elements of 0^. 

Lemma 6.6 The set of leading exponents of a joint eigenfunction 0/O1, ^2 in the bundle V(j)<8> 
V{pf) with generic eigenvalue z is 

{P(.,e),(*',e>)( z ) I a G M(ry),<r' £ M( 7 ), e, e' £ {+, -}}, 

where P( a ,e),(a' ,e')( z ) = ( — P + e ^a( z )i — P + e '^u'{ z ))- The corresponding boundary value is a 
section of the bundle V(^y(a) e x lT (z)) ® V{l{°~')t'\ a i(z)) ~^ 9X x dX, where 7(a), 7(0"') denote the 
isotypic components. 

Proof. An eigensection of Q in V{^) — > X has leading exponents — p + eA^^), e 6 {+, — }, and 
the corresponding boundary value is a section of ^(7(o") e A CT (z))- This implies the lemma. □ 

Let <9A C dX x <9X be the diagonal in the boundary and define dS := (Ji^ 7 r(7 x 1)(^A n 
f2 x fi). Note that d{z) is a joint eigenfunction in a neighbourhood inXxJofr!xQ \ dS. 
Therefore, for generic z it has boundary values along this set §§. We denote the boundary value 
associated to the leading exponent r := P( a . e ).(a' ,e')( z ) by /3 T (f). 



6.3 Boundary values 
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Lemma 6.7 We have (3 T {d(z)) = (the meromorphic family of hyperfunctions vanishes) except 
for t = [j,r a \jfrw t _)(z) ; o £ M("f), in which case (3 T {d{z)) is a meromorphic family of real 
analytic sections. 

Proof. We employ the fact that j3 T {d(z)) depends meromorphically on z. Let U C SI be such that 
the restriction of the projection 0, — > B is a diffeomorphism. There is a constant c > such that 
for all £2) G U x f7 we have e rf °( fciai ' fc2a2 ) > c| max(ai,a2)| (see ||, Cor. 2.4). Using Lemma 
|6li| we see that for Re(z) < 6 we have \d(z)(kiai, &2d2)| < C| max(ai, 02) I _ v /f,_Re ( z ) ) where C 
depends on z. If one of the signs e, e' is positive, for z -C we have \/6 — z — 2p + eA<j(z) + 
e'A<j/(z) > 0. For those z we have lim min ( aii(I2 )_ >00 d(z)(&iai, fc2«2)(&l> ^2) ~ M K e ).( CT '. E ')( 2 ) = 
uniformly in (ki,kz), where (a,bp^' u ' := 0^6^. This shows that I3 r {d{z)) = if one of e,e' is 
positive. 

We now consider the kernel r(z) on I x I \ A. It is a joint eigenfunction of f2i, ^2 to the 
eigenvalue z on a neighbourhood of <9X x \ <9A and therefore has hyperfunction boundary 
values along this set. A similar argument as above but using Lemma [T4] instead of shows 
that f3 T (r(z)) = except for e = e' = — . 

Note that r(z) is G-invariant in the sense that for g £ G we have L g ® L g r(z) = r(z). If 
T = M(a-,-),(o-',-)(< 2; )> then /3 r (r(z)) is a G-invariant hyperfunction section of V( r y(a)_x !r (z)) ® 
^"(tC^O-a /, J over x 9X \ dA. Since this set is an orbit of G, an invariant hyperfunction 
on this set is smooth, and the evaluation at the point (w, 1) S dX x dX provides an injection 
of the space of invariant sections into V^(cr) ® Vy(cr'). If b is such a G-invariant section, then we 
have for ma 6 MA 

= b(maw,ma) 
= b(wm w a~ 1 ,ma) 

Thus b(w, 1) € [Ky»((T w ) <8> V~(ct')] M - We conclude that o 7 ^ ct™, and in this case A CT /(z) = A (T (z) 
holds automatically. Thus A-(r(z)) = if cr' ^ d w . 
We write 

F( 7 (<7)-a) ® V(7(cr«)_ A ) = V>_ A ) ® y(cr« A ) ® Hom M (K,^ 7 ) ® Hom M (U^,V~) . 

The space of invariant sections of V a _ x (8) ^o-™ A over x <9X \ 5 A is spanned by the distribution 
kernel j™w of the Knapp-Stein intertwining operator J™w. We conclude that for each a E M(j) 
there is a meromorphic family A a (z) £ Horn^T^, Vy) (8 Homjw(Vg-«>, V^) such that for r = 
/"(o-,-),^™,-)! 2 ) we have (3 T {r(z)) = j^, (8) A a (z) under the identifications above. 



38 



6 RESOLVENT KERNELS AND SELBERG ZETA FUNCTIONS 



Let r = fj,r a \rfrw\(z). We now employ Lemma [3j| which states that for Re(z) « we 
have d(z) = Xa^7er(-^7 ® l)r(z). The sum converges locally uniformly and thus in the space 
of smooth section over X X X \ S. We further see that convergence holds locally uniformly in a 
neighbourhood of ft x ft \ dS. Thus by we can consider distribution boundary values, and 
by continuity of the boundary value map we have on ft x ft \ dS 

(3 T (d(z)) = ^ (7r^W( 7 )8)l)/3 T (d(^) 

= ( vrCT ~ ACT(z) W ® i^.A.w ® 

where s w „, , , * is the distribution kernel of the scattering matrix S w ,„ , , Here we use the 
identity 7r* o J^, a , , = 5^, A o 7T* which implies that the distribution kernel of the scat- 
tering matrix S™ w -> can be obtained by averaging the distribution kernel of the Knapp-Stein 
intertwining operator J™ w A for Re(A) S> 0. 

It follows from the results of |3] that — & extends to a meromorphic family 

A CT (z) A CT (z) 

of smooth sections on all of a£. It follows from 0, Lemma 2.19, 2.20, that it is indeed a 
meromorphic family of real analytic sections. Strictly speaking, in ||] we only considered the 
spherical M-type for G = SO(l, re), but the same arguments can be applied in the general case. 
We conclude that f3 T (d(z)) is real analytic as required. 

A similar reasoning shows that (3 T {d{z)) = for all r which are not of the form ^( a ,-),(a w ,-)i z ) 
for some a E M(y). □ 



Lemma 6.8 We have an asymptotic expansion (for generic z) 

oo 

d(z)(k l a,k 2 a) a ~ 00 J2 Y, a ' 2p ~ 2K{Z) ~ n >^{k 1 M) (45) 
<reM( 7 ) n=0 

which holds locally uniformly for k £ ft x ft\dS, and where the real analytic sections p z ,a,n(ki, fe) 
of V( / y(a)^\) (g) V( , y(a w )_\) depend meromorphically on z. 

Proof. Since the boundary value of d(z) along ft x ft \ dS is real analytic we can employ |10|| , 
Prop. 2.16, in order to conclude that d(z) has an asymptotic expansion with coefficients which 
depend meromorphically on z. The formula follows from an inspection of the list of leading 
exponents Lemma |6.7| . □ 



6.4 The regularized trace of the resolvent 
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Lemma |6^ has the following consequence. For generic z we have 

oo 

tr d{z){kaM) a ~^ E Y. a ~ 2( " 2X ° [Z) ~ na P^{k) 
o-eM( 7 ) ™=° 

which holds locally uniformly for k G f2, and where the real analytic functions p Z:(T ,n depend 
meromorphically on z. 

6.4 The regularized trace of the resolvent 

Lemma 6.9 The integral Qj(z) := J x x r (x)tr d(z)dx converges for Re(z) <C and admits a 
meromorphic continuation to all o/C 7 . 

Proof. Convergence for Re(z) <C follows from Lemma |6.3| . Fix R G A. We write Q 1 {z) = 
Qi(z,R) + Q 2 (z,R), where 

Qi{z,R):= I I x T (ka)tr d{z){ka)dkv{a)da , 
Ji Jk 

where v is such that dkv(a)da is the volume measure on X. Note that v(a) ~ a 2p {uj x + oT a c x + 
a~ 2a C2 + ...) as a — > oo. 

It is clear that Qi(z, R) admits a meromorphic continuation. We have an asymptotic expan- 
sion as a — > oo. 

oo 

u(z,a):=l X T (ka)tvd(z)(ka)dkv(a) ~ £ £ ar 2X "^- na q z ^ n , 
X CT eA/( 7 ) n=0 

where q depends meromorphically on z. For m G N let 

m 

u m (z, a) := u(z, a) - E E a ~ 2K{z) ' na< lz,a,n ■ 

Given a compact subset W of C 7 we can choose m G No such that u m (z, a) da converges (for 
generic z) and depends meromorphically on z for all z G W. We further have 



EE/ a-^W—gwft, = E E 



^ R ~2\ a (z)-na qz 



2\ a (z) + na ' 

(reM(7) n =° aeA/( 7 ) n = 

and this function extends meromorphically to C 7 . Since we can choose W arbitrary large we 
conclude that Q-y(z) admits a meromorphic continuation to all of C~. □ 
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6.5 A functional equation 

Let £(7) := {co- I a G M(j)} be the set of ramification points of C 7 , define C" := C \ £(7), and 
let C 7 C C 7 be the preimage of under the projection C 7 — ► C. Then C 7 — > is a Galois 
covering with group of deck transformations IT := ©^(7)^2- The action of II extends to C 7 such 
C 7 \ C 7 consists of fixed points. For I £ £(7) let q\ G II be the corresponding generator. Then 
we have X a (qiz) = —X a (z) for all a E ^(7) with c a = I and X a i(qiz) = X a i(z) else. 

Lemma 6.10 For I £ £(7) we have 

Q y ( qi z) - Q 7 (z) = £ Z^-Zr^W) . 

o-eM(7),c CT =« 



Proof. In the proof of Lemma |6.1| we have seen that (using the notation introduced there) 
We conclude that 

CTGM(7), CtT =« 3 

The same reasoning applies to the trivial group T, where the Eisenstein series get replaced by 
the Poisson transformations. Thus we can write 

d{q lZ ) - d(z) = £ £ Opg W ^ _ 1} ( 0p^W )# 

<TSM(7),c CT =i ff i 



The proof of Lemma 6J3 shows that Q\(z, R) has an asymptotic expansion 

00 p-2A CT (z)-na n 

Q 1 (z J R)~Q J (z)+ £ X T . 

<reM(7) n = 

In particular, if 2Aq-(z) E" — No« for all a, then Q 7 (z) is the constant term in the asymptotic 
expansion of Qi(z, R). 



We can now apply Proposition 4.4 which can be interpreted as the determination of the 
constant term (as R — > 00) of 

/ x T (ka)tr o (ear* o tt* - 1) o (°P^ (a) )*l ka)dkv(a)da 

l J K ^ - 1 

as a distribution on ia* \ {0}. This shows the desired equation first on z a {ia*) and then every- 
where by meromorphic continuation. □ 



6.6 Selberg zeta functions 
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6.6 Selberg zeta functions 

For a detailed investigation of Selberg zeta functions associated to bundles (for cocompact T) 
we refer to 0. We assume that a is irreducible and Weyl invariant, or that it is of the form 
<?' © ( cr ') w f° r some non-Weyl invariant irreducible M-type a' . In the latter case we define 
Lr(7r CT < A ) := L r (vr°"'' A ) + L r (>') W ' A ). 

Let P = MAN be a parabolic subgroup of M. If 7 G V, then it can be conjugated in G to 
an element m g a g € MA with a g > 1. Let n be the negative root space of (fl, a). For Re(A) > p 
we can define the Selberg zeta function Zs(X,o~) by the converging infinite product 

00 

Z s (X,a):= 11 ]]det(l-a(m g )®S k (Ad(m g a g ) lh )a g x -^ . 
i^[ g ]ecrfc=o 

In the case of cocompact T it was shown by || that Zs(X,a) has a meromorphic continuation 
to all of a* c . In (T§ it was explained that the argument of ||] extends to the case of convex 



cocompact subgroups since it is the compactness of the non- wandering set of the geodesic flow 
of Y that matters and not the compactness of Y. Strictly speaking, [|ll]] deals with the spherical 
case of SO(l, 2n), but the argument extends to the general case. 

There is a virtual representation 7 of K (i.e. an element of the integral representation ring 
of K) such that 71^ = a in the integral representation ring of M (see |J, 0). We call 7 a lift of 
a. Note that 7 is not unique. We can extend the material developed above to virtual ET-types 



by taking the traces with corresponding signs. Because of the factor [7 : a] Lemma 6.1C has the 
following corollary. 

Corollary 6.11 Ifj is a lift of a, then Q 7 (z) extends to a twofold branched cover ofC associated 
with \a(z). 



Theorem 6.12 The Selberg zeta function satisfies 



Z s (\,a) 

- exp 



f L r (vr CT ' A )dA 
J 



Z S (-X,a) 

In particular, the residues o/Lr( 7r ' 7 ' A ) ar & integral. 

Proof. By Q Prop. 3.8. we have 

QiM*)) = ^Z' s (X,a)/Z s (X,a) (46) 

for Re(A) 3> 0. Indeed, Qy(z (T (X)) is just what is called in the hyperbolic contribution 
associated to the resolvent. So Corollary |6. 11 and Lemma S.10| yields the functional equation of 
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the logarithmic derivative of the Selberg zeta function 

Z' s i-\<y) , Z' s (\a) A 
Z s (-\a)^ Z s (\,a) F{ ] - 

Integrating and employing the apriori information that Zs(\,cr) is meromorphic we obtain the 
desired functional equation. □ 

Remarks : 

1. As explained in the introduction it is known (from the approach to Z$ using symbolic 
dynamics and Ruelles thermodynamic formalism) that Zs(X) is a meromorphic function 
of finite order. It follows that Lr( 7r<J,A )) as a function of A, grows at most polynomially. 

2. In order to describe the singularities of Z$(\, cr) we assume that 7 is an admissible lift of 
a (see 0). Let n\ >a denote the (virtual) dimension of the subspace of the L 2 -kernel of 
Qy — z a (\) on Vyiri) which is generated by non-discrete series representations of G. It 
follows from Theorem |6.12 that 

' res A=At L r (vr <T ' A ) + n_ Mj0 . Re(/x) < 
n^a < Re(/i) 



ord A=/ ,Zs(A,cr) 



3. If n is non-integral and if ext has a pole at \x of at most first order, then one has 

res A= ^L r K' A ) = dim r C7-°°(A, V(a,fj)) , 

where r C~°°(A, V(a, //)) is the space of invariant distributions with support on the limit 
set A. If ext has higher order singularity, then the residue has a similar interpretation (see 
0], Prop. 5.6) This provides an independent argument for the integrality of the residues 

Of L r (TT a ' X ). 

6.7 Integrality of Nr(ir) for discrete series representations 

Let 7T be a discrete series representation of G containing the -ftT-type 7. There are embeddings 
M^KCr) ^ L 2 (r\G)( 7 ) and V*®V v (?r) ^ L 2 {G). Let A G End^(K(7)) be given. We extend 
A by zero to the orthogonal complement of ^(7) thus obtaining an operator in End^V^-) which 
we will still denote by A. The operator A induces operators Ry^a) and Rr(tiA) on L 2 (r\G) 
and L 2 (G), where Ha is supported on {tt} C G and /ia(tt) := A. 

Lemma 6.13 We have 

K kv{hA) {g,9)-K k{hA) {g,g)eL\T\G) . 



6.7 Integrality of Nr(ir) for discrete series representations 
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Proof. Let D(G,j) be the algebra of invariant differential operators on V(7). It is isomorphic 
to (U(q) ®u(t\ End(V^)) . If it' is an admissible representation of G, then D(G, 7) acts in a 
natural way on (V n i ® Ky) . If vr' is irreducible, then (V^i Vy) K is an irreducible representation 
of D{G, 7). The correspondence 7r' 1— > (V^ (8> V^) provides a bijection between the sets of 
equivalence classes of irreducible representations of G containing the .fT-type 7 and irreducible 
representations of D{G, 7). 

Note that End^V^)) = End((K (8) V^). We conclude that there is D A £ D(G, 7) that 
induces the endomorphism A on ^(7). Let zo be the eigenvalue of the Casimir operator on tt 
and Z be the finite set of irreducible representations of G containing the -KT-type 7 such that O 
acts with eigenvalue zq. Then we can choose Da such that it vanishes on all (V n i V^) K for 
n' e Z, ttV tt- 

For simplicity we assume that zq is not a branching point of R{z). In the latter case the 
following argument can easily be modified. The operators DaRy(z) and DaR(z) have poles 
at zq with residues K^^^ and K^ hA y The difference (Da)ioI(z) := (Da)iTy(-z) — (-t^OiK^) 
of distribution kernels is still a meromorphic family of joint eigenfunctions with real analytic 
boundary values along fixfl \ dS. We have the asymptotic expansion 

00 

er6M(7) n=0 

The residue of (DA)id(z) at zo can be computed by integrating {Da)\oI{z) along a small circle 
counter-clockwise surrounding zq. If we insert the asymptotic expansion ( f47| ) into this integral, 
then we obtain an asymptotic expansion 

00 

ve Sz=Z0 (D A )id(z)(k ia ,k 2 a) a ~°° £ ^ res^a^-^^-^^A^, fc 2 ) 

o-eM( 7 ) n=0 

00 finite 

~ E E E log(«) m ^ 2 ^ 2Aff(2oKn Vo,a,n, m ,A(A;i,fc 2 ) , 

where p Zo ,cr,n,m,A is a real analytic section on $7 x $7 \ 95. Since (ka an d ^R{h A ) project 
onto eigenspaces of square integrable sections we have for k\ / k 2 

K R(h A )( k i a i> k 2 a i) ai ~°° 

00 finite 

£ E lo g(°i) mi log(a 2 )^a^- ACT ^ ) - niQ a^- ACT ^ ) - n2a p Z0 , CT , ni , n2 , mi , m2 ,A(A; 1 , fc 2 ) 

ni,ri2=0 mi,rri2 

withp 20)(T)ril)n2)mi)m2) A(A;i, & 2 ) = as long as -A CT (z ) - ma > 0, -A CT (z ) - n 2 a > 0, and similar 
for ^R r fh A )- We conclude that Re(— 2\ a {z) — na) < if p ZOtCT ,n,m,A 7^ 0. The assertion of the 
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lemma now follows. □ 



Proposition 6.14 If tt be a representation of the discrete series ofG, then Nr(jr) G Z. 

Proof. There exists an invariant generalized Dirac operator D acting on a graded vector bundle 
E — » X, E = E + © E~, such that Va © {0} is the kernel of D. If 7 is the graded K-type 
associated to E, then 

7| M = . (48) 

Let D Y be the induced operator on Y. The distribution kernels of r{z) := (z — D 2 )^ 1 and 
r Y (z) := (z — Dy)^ 1 have meromorphic continuations to a branched covering of C. Their 
difference goes into the functional The function 



Q{z):= I tr (r Y (z) - r(z)) (g, g)dg 
lr\G 



has a meromorphic continuation to all of C by (48) and Lemma 6.1C. Its residue at z = is 
given by 

res z=0 Q(^) = A^r(vr) + ^ n(j,Tr')Nr(n') , 

7T'GG C 

where the sum reflects the fact that a finite number of representations belonging to G c may 
contribute to the kernel of Dy- Note that 11(7, tt') G Z and Nr(ir') £ No- We show that 
res z= o(5(-2;) = in order to conclude that iVr(7r) = — X^r'eG n(-y,Tr')Nr(7r') G Z. 

It suffices to show that Q(z) = for Re(z) <C 0. This follows from (|46|), but we will give an 
independent argument. We can write 

r Y (z) - r(z) = - z {{D Y ) x r Y {z) - (D 2 )^)) . 

Integrating the restriction of this difference to the diagonal over T\G we obtain 



Q(z) = -f tr ({D Y ) 1 ry(z)-(D 2 ) 1 r(z))(g,g)dg 
~ IT\G 



Z 



- [ x r (g)t? ((Dy)iry(z) - {D 2 ) l r{z)){g,g)dg 
z Jg 

-- / X T (g)^({DY)i(Dy) 2 ry{z)-D l D 2 r(z)){g,g)dg 
z JG 

-- [ tr c(dx r )i((D Y ) 1 r Y (z)-D 1 r{z))(g,g)dg 

z JG 



1 



X r (<7)tr ((D Y )ir Y (z) - D 2 r(z))(g, g)dg 



z Jg 

- [ trc(dx r )i((D Y hr Y (z)-D 1 r(z))(g, 
z JG 
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where c(dx r ) denotes Clifford multiplication. We conclude that 



Q( Z )=-1T [ trc(dx r )i((D Y )ir Y (z)-D 1 r(z))(g,g)dg . 
Ig 



2z 



The right hand side of this equation vanishes as a consequence of S 7 er^*X r = ^ anc ^ * ne ^- 
invariance of ((Dy)\Ty{z) — D\r(z))(g, g). This finishes the proof of the proposition. □ 
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